Check the input once in pred_param and skip trees that never split - #462

Reopening #452, which I closed by mistake; GitHub could not reopen it because the source fork had been deleted. The change is identical.

What this improves

Predictions with the default tree learners are about 2x faster, and the results are bit-identical to before. Most of the time in pred_param went to scikit-learn validating the same input once per tree (1,000 checks for a 500-round Normal model), not to walking the trees. Models whose trees never split now predict in under a millisecond, and fit warns when that happens, because such a model returns the same distribution for every input.

How

• When every base learner is a DecisionTreeRegressor or ExtraTreeRegressor, pred_param checks X once (float32, only the columns the trees read, NaN allowed only where every tree allows it, as each tree's own check does) and calls the trees with check_input=False.
• A tree whose leaves all hold the same value is replaced by its scalar update. Updates before a parameter's first real split are folded into the start values. Later ones stay in round order. Each parameter goes through the same floating-point operations in the same order as the loop, so the output is bit-identical.
• The precomputed plan keeps references to the rounds it was built from. It is rebuilt if base_models, scalings, col_idxs, init_params or learning_rate change (refit, partial_fit, edits by hand), so it cannot go stale. It is left out of pickles and rebuilt on the first prediction.
• max_iter, sparse input and non-tree learners (for example Ridge) still use the existing loop.
• fit and partial_fit raise NGBoostDegenerateWarning when every learner is constant. This happens without any message when int(minibatch_frac * n) is below max(min_samples_split, 2 * min_samples_leaf).

Proof it works

• New tests/test_prediction_plan.py (16 tests) compares pred_param bit for bit with the old loop. It covers all-constant, mixed and trailing-constant models, Normal, T, Bernoulli and k_categorical(3), minibatching, col_sample, early stopping, max_iter, staged predictions, refit, partial_fit, edited rounds, pickling, ExtraTreeRegressor, monotonic_cst, missing values, wide and narrow input, Ridge and sparse input. It also checks the warning.
• Full suite with --slow: 185 passed, 2 xfailed, 22 xpassed. black, isort, flake8 and pylint are clean (10.00/10).
• Timing, median pred_param time before and after on Python 3.12, scikit-learn 1.6.1 and numpy 2.5 (one shared 8-core machine). Every output was bit-identical:

References

Related to #298 (a request for faster NGBoost training and inference). I found no open issue or PR for this change.

AI disclosure

This change was written with help from an AI coding assistant (Claude). The tests and linters listed above were run on this branch before opening the PR.